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Introductory  demonstration 

Most  everyone  has  had  the  experience  of  trying  to  give  cough  syrup  to  a  small  child. 
There  is  a  whole  ritual  to  this,  involving  the  assembling  of  the  materials,  little  tricks 
for  keeping  the  drop  of  medicine  that  always  remains  on  the  lip  of  the  bottle  from 
falling  off,  and  cajoling  on  the  order  of  you’re  a  big  girl,  now,  aren’t  you ?  Giving 
cough  syrup  to  a  small  child  is  an  example  of  a  routine.  The  most  memorable  part 
of  the  cough  syrup  routine  is  the  point  at  which  the  brimming  spoon  of  cough  syrup, 
kept  from  spilling  only  by  its  surface  tension,  invariably  spills  just  before  entering  the 
child’s  mouth.  This  is  a  hassle,  something  that  regularly  goes  wrong  in  the  course 
of  a  routine.  Both  the  routine  and  the  hassle  are  institutions,  in  the  sense  that  the 
experiences  they  reflect  are  nearly  universal  in  our  culture  even  if  they  are  rarely 
articulated.  In  the  back  of  everyone’s  mind  is  a  wish  corresponding  to  each  hassle.  A 
wish  is  a  mechanism  that  combs  through  everyday  experience  looking  for  the  answer 
to  some  routine  question  or  the  solution  to  some  routine  problem. 

I  have  formulated  these  ideas  by  a  method  of  systematic  introspection.  I  ob¬ 
serve  regularities  in  the  way  that  ordinary  people  deal  with  everyday  life  and  make 
computational  theories  to  explain  these  regularities.  Then  I  use  the  terms  of  the 
computational  theories  as  an  observation  vocabulary  for  introspecting  on  everyday 
activities.  For  example,  picture  the  moment,  when  trying  to  give  cough  syrup  to  a 
small  child,  that  you  accidentally  spill  it.  Suppose  I  tell  you  there  is  a  new  product 
on  the  market,  cough  syrup  in  gel  form,  whose  makers  advertise  that  it  “never,  ever 
spills”.  That  something  that  goes  click  in  your  head  is  a  wish  being  triggered.  Com¬ 
putation  provides  an  observation  vocabulary  for  introspection.  This  paper  is  a  sketch 
of  a  theory  I  developed  by  a  principled  method  of  alternating  between  introspection 
and  theory  construction. 

Routines  and  self-models 

Routines  are  the  frequently  repeated  and  phenomenologically  automatic  rituals  of 
which  most  of  daily  life  is  made.  Routinization,  a  term  borrowed  from  business 
management  (see  Chandler  1962),  is  the  process  by  which  institutionalized  patterns 
of  interaction  with  the  world  arise  and  evolve  in  everyday  life.  Routinization  is  hard 
to  study  because  routines  arise  with  no  discernable  effort  and  because  most  changes 
to  mundane  routines  are  at  most  barely  conscious.  You  can  observe  an  especially 
graphic  example  of  routine  evolution  by  stuffing  a  few  thousand  envelopes.  I  (and 
others)  have  observed  that  the  envelope-stuffing  novice’s  technique  changes  with  time, 
one  change  every  few  dozen  envelopes.  The  first  few  envelopes  will  be  stuffed  in  the 
“obvious”  way.  But  then  as  you  have  the  opportunity  to  observe  the  process  of  stuffing 
each  envelope  and  moving  on  to  the  next,  you  will  notice  small  opportunities  to  use 
your  fingers  more  conveniently,  pick  up  the  envelopes  more  reliably,  fold  the  stuffing 
more  efficiently,  and  so  on.  After  several  hundred  envelopes  have  gone  by,  the  original 
technique,  which  arose  based  on  only  the  most  basic  properties  of  the  task,  will  have 
evolved  into  a  technique  that  takes  account  of  a  lot  of  very  subtle  properties  of  the 
envelopes,  stuffing,  stamps,  and  workplace,  not  to  mention  of  your  hands. 
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In  trying  to  Gnd  computational  terms  to  describe  the  process  of  routinization  in 
everyday  life,  I  have  been  led  to  the  following  formulation  of  the  central  weakness 
of  both  the  classical  planning  literature  (Sussman  1D75,  Sacerdoti  1977,  etc)  and  the 
more  recent  work  on  reflexive  (“meta- level”)  systems  (Lenat  1983,  Genesereth  1983, 
Smith  1982,  etc): 

In  order  to  sensibly  modify  itself,  an  intelligent  being  must  have  a  good  rep¬ 
resentation  of  itself,  a  self-model.  Plans,  rules,  heuristics,  and  the  like  are 
representations  of  procedure.  Self-models  based  on  procedures  imply  a  separa¬ 
tion  between  inner  and  outer  reality  that  is  inappropriate  for  beings  so  deeply 
embedded  in  the  world  as  ourselves.  Instead,  the  bulk  of  an  intelligent  system's 
self-model  ought  to  be  expressed  in  terms  of  common  processes  of  interaction 
between  itself  and  its  world1 2. 

The  difference  between  procedure  and  process  self-models  is  especially  evident 
when,  as  often  happens,  there  is  a  great  abstract  gap  between  what  you  think  you’re 
doing  and  the  most  useful  phrasing  of  the  process  you’re  actually  engaged  in.  An 
extreme  example  of  this  gap  is  provided  by  Simon’s  Ant  (1970),  whose  behavior  is 
determined  by  a  very  simple  mechanism  that  interacts  with  the  great  complexity  of 
its  environment  to  produce  a  complicated  path.  If  this  path  were  to  lead  the  ant  in 
circles,  its  inability  to  notice  or  represent  this  fact  would  leave  it  trapped.  An  everyday 
example  is  provided  by  the  myopic  vacuumer  of  dining  rooms  who  hasn’t  thought  to 
describe  the  process  as  one  of  alternating  between  vacuuming  and  furniture-moving. 
Explicitly  representing  that  aspect  of  the  vacuuming  process  should  make  one  think 
to  move  all  the  furniture  before  getting  started. 

If  people  behaved  like  classical  planners,  developing  a  full-blown  plan  off-line  before 
doing  anything,  the  difference  between  procedure  and  process  self-models  wouldn’t 
be  important.  But  the  complexity  and  uncertainty  of  new  situations  generally  makes 
this  impossible.  Instead,  people  improvise*.  Improvised  activity  very  frequently  has 
patterns  to  it  that  are  nowhere  represented.  Routinization  is  the  internalization  of 
process:  you  build  models  of  your  existing  patterns  of  interaction  with  the  world  and 
use  these  models  to  monitor  and  (when  possible)  to  improve  the  processes*. 

Routines,  I  emphasize,  are  not  procedures.  A  procedure  is  a  description,  that  is, 
a  data  structure,  but  a  routine  is  an  abstract  entity  in  the  world  that  exists  prior 

1  Doyle’s  (1980)  assimilation  of  thought  to  action  speaks  to  this  intuition  but  is  only  half  an  answer. 
Procedural  representations  of  action,  whether  the  nction  is  inner  or  outer,  describe  only  inner  reality. 

2  Suchman  (1984)  contrasts  plant  and  situated  aetion.  Suchman  “treats  plans  as  derivative  from  situ¬ 
ated  action.  Situated  action  as  such  comprises  neccssiirily  ad  hoc  responses  to  the  actions  of  others 
and  to  the  contingencies  of  particular  situations.”  Her  central  concern  is  the  mutual  intelligibility  of 
activity.  “Rather  than  depend  upon  the  reliable  recognition  of  intent,  successful  interaction  consists 
in  the  collaborative  production  of  intelligibility  through  mutual  access  to  situation  resources,  and 
through  the  detection,  repair  or  exploitation  of  differences  in  understanding.” 

*  Sartre’s  (1983)  distinction  between  praxit  and  process  is  representative  of  European  attempts  to 
base  philosophies  of  political  praxis  on  the  internalisation  of  theories  of  political  process.  (See 
Bernstein  1971  for  a  survey.)  Laing  has  taken  this  vocabulary  over  to  a  psychiatric  setting  (sec 
Laing  and  Cooper  1971);  his  primary  concern  is  the  destructive  processes  within  families  (see  Laing 
and  Estcrson  1970). 
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to  any  model  of  it.  A  procedure  determines  a  certain  vocabulary  of  description,  but 
a  routine,  like  anything  else  in  the  world,  will  in  general  admit  of  many  different 
models4. 

Modifying  a  maladaptive  routine  pattern  of  activity  is,  in  general,  a  very  difficult 
problem.  Often  each  of  the  separate  decisions  that  enter  into  an  interaction  make 
sense  individually,  even  though  the  result  involves  annoyance,  danger,  excess  effort, 
or  outright  failure.  In  most  cases,  an  observer  would  ascribe  the  problem  to  a  lapse 
of  foresight.  We  can  identify  three  problems  in  the  modification  of  a  routine: 

•  Noticing  that  something  is  wrong,  or  that  something  could  be  better,  can  be 
cither  easy  or  hard.  When  you  cut  yourself,  you  know.  But  it  is  difficult  in 
genera]  to  notice  that  a  different  assignment  of  tasks  to  your  two  hands  would 
allow  some  of  them  to  be  performed  in  parallel.  There  is  no  substitute  for  an 
extensive  vocabulary  of  interesting  properties  of  processes,  and  a  (presumably 
highly  parallel)  procedure  for  noticing  instances  of  them. 

•  Intervening  in  an  existing  routine  decision  process  can  be  difficult  if  the  wrong 
decisions  are  not  connected  to  the  symptom  in  any  readily  obvious  way.  An 
important  case  of  this  problem  is  when  you  have  forgotten  how  you  make  some 
routine  decision,  and  it  has  become  a  habit.  Occasionally  a  complete  reformulation 
of  the  way  you  think  about  the  problem  is  called  for. 

•  Anticipating  the  point  in  a  routine  at  which  an  intervention  is  required  can  be 
difficult  when  nothing  reminds  you  that  a  problem  is  coming  up.  It  is  a  common 
experience,  when  driving,  to  routinely  forget  to  put  on  your  sunglasses  until  you’re 
out  on  the  road  and  retrieving  them  from  the  glove  compartment  becomes  difficult. 

There  is  no  general  solution  to  any  of  these  problems.  There  is,  instead,  a  collection 
of  generic  solutions  to  special  cases  of  them. 

Classification  of  changes  to  routines 

There  are,  then,  two  main  trends  in  the  evolution  of  a  routine.  (1)  The  routine  takes 
increasingly  detailed  account  of  the  particulars  and  peculiarities  of  the  environment  it 
is  carried  out  in.  (2)  The  routine’s  owner  acquires  an  increasingly  detailed  and  global 
array  of  models  of  the  process  of  carrying  it  out. 

I  find  it  useful  to  distinguish  five  classes  of  incremental  changes  a  routine  can 
undergo5.  The  classes  are  characterized  by  increasingly  more  sophisticated  ways  of 
internalizing  processes  in  the  routine’s  execution. 

4  1  depart  from  tradition  in  using  the  words  representation,  description,  and  model  to  refer  to  the 
same  idea,  roughly  Minsky’s  notion  cl  a  frame  (1975).  For  further  discussion,  see  the  section  on 
lattices. 

5  Fitts  and  Posner  (1907)  and  Schneider  (1984)  also  present  theories  of  the  stages  by  which  activities 
become  automatic.  They  are  not  concerned,  however,  with  the  evolution  of  a  routine,  just  with 
its  becoming  automatic.  Moreover,  their  models  of  the  architecture  erf  the  inind  arc  so  radically 
different  from  mine  that  there  is  no  basis  for  comparison  of  the  different  theories. 
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(1)  Recapitulated  reasoning.  If,  on  two  occasions,  you  approach  the  same  world 
with  the  same  goal  and  the  same  ideas  about  how  to  achieve  it,  then  you’re  going 
to  take  the  same  action  both  times.  I  hypothesize  that  your  mind  stores  away  a 
description  of  the  process  of  reasoning  that  led  to  taking  that  action.  By  means  of 
this  description,  recognizing  that  the  world  (both  inner  and  outer  reality)  once  again 
satisfies  the  premises  of  an  earlier  line  of  reasoning  should  automatically  lead  to  the 
same  conclusion.  This  is  the  idea  behind  Doyle’s  Reason  Maintenance  System  (1980). 
If  you  have  no  desire  to  do  things  differently,  the  routine  will  stop  evolving  at  this 
point. 

(2)  Primitive  credit  assignment.  Unpleasant  experiences  are  the  primary  motor  of 
routine  evolution.  At  first,  finding  a  decision  to  blame  for  an  unpleasant  experience  is 
governed  by  a  single  primitive  heuristic:  when  making  an  underconstrained  decision, 
look  around  quickly  (in  the  world  or  your  memory)  for  evidence  of  a  bad  thing  that 
previous  choices  at  this  point  led  to  (as  determined  by  domain  dependent  reasoning). 
The  prototype  is  the  puddle  you  stepped  in  yesterday  on  the  high  road.  Arriving  at 
the  fork  in  the  road  today,  if  you  notice  the  puddle  up  ahead  on  the  high  road,  you 
might  take  the  low  road  instead.  But  when  you’re  deciding  which  road  to  take  on 
the  return  trip,  there  might  not  be  any  ready  reminder  of  the  puddle.  This  is  why, 
for  many  people,  the  routine  paths  back  and  forth  between  two  points  are  often  not 
reverses  of  one  another. 

(3)  Scene  characterization.  The  first  significant  process  representation  is  the  script 
(Schank  and  Abelson  1975),  organized  as  a  simple  chain  of  scenes.  Many  opportunities 
for  routine  improvement  can  be  noticed  by  characterizing  individual  scenes.  A  scene  in 
which  you’re  waiting  for  something  might  suggest  moving  useful  work  into  the  waiting 
period,  as  when  you  put  the  water  on  to  boil  before  getting  out  the  tea.  Waiting  is  a 
generic  hassle.  As  I  explain  below,  generic  hassles  motivate  generic  repairs. 

(4)  Process  characterization.  Many  common  patterns  more  global  in  scope  than 
a  single  scene  also  suggest  improvements  to  routines.  Doing  something  and  then 
undoing  it  can  suggest  omitting  both  steps.  Often  you  can  take  advantage  of  some 
coincidence  to  replace  two  actions  that  you  happen  to  take  successively  with  a  single 
action.  Such  optimizations  are  remarkably  common  when  the  routine  first  arose  by 
recursive  decomposition  of  generic  plans.  Vacuuming  is  often  best  characterized  as 
interleaving  the  processes  of  moving  the  furniture,  running  the  vacuum  around,  and 
returning  the  furniture.  Sometimes  it  is  better  to  dcinterleave  these  processes. 

(5)  Process  manipulation.  In  engineering  and  business,  one  often  constructs  and 
manipulates  formal  models  of  processes,  attempting  to  apply  powerful  analytical 
methods  to  discover  the  optimum  procedures.  Clear  cases  of  such  reasoning  is  com¬ 
paratively  unusual  in  everyday  life,  though  it  is  present  in  the  making  of  diagrams 
in  activities  like  navigation.  People  sometimes  do  carry  out  the  complex  debugging 
skills  embodied  in  programs  like  Hacker  (Sussman  1975),  and  parts  of  that  reasoning 
should  surely  be  assigned  to  this  class*. 

*  The  work  of  VanLehn  (1983)  and  Sussinan  (1975)  on  plan  repair  is  relevant  here,  but  I  emphasize 
that  I  am  not  concerned  with  the  debugging  of  faulty  plans  but  rather  with  the  improvement  of 
established  routines.  Diagnosing  and  repairing  a  hug  in  a  plan  requires  a  much  cleaner  and  more 


Generic  models 


Representations  reflect  regularities  in  the  world.  Routines  reflect  regularities  in  the 
world  too,  but  much  more  implicitly,  through  regularities  in  the  way  you  deal  with 
the  world.  This  parallel  suggests  some  important  interactions  between  the  theory  of 
routines  and  the  theory  of  representation.  These  interactions  fall  into  two  classes,  (1) 
the  ways  in  which  the  organization  of  knowledge  supports  the  process  of  routiniza- 
tion,  and  (2)  the  ways  in  which  the  regularities  that  routinization  stumbles  onto  are 
internalized  as  new  domain  representations.  I  will  concentrate  on  (1)  here. 

I  imagine  the  knowledge  supporting  routinization  to  be  organized  in  a  frame  net¬ 
work  (Minsky  1975).  I  will  introduce  my  ideas  about  the  structure  of  this  frame 
network  with  a  discussion  of  generic  hassles.  Suppose  you  didn’t  have  the  idea  of 
waiting  in  your  head.  You  could  spend  ten  minutes  every  morning  of  your  life  staring 
into  space  until  the  water  boils  without  ever  clearly  formulating  what  is  bothering 
you.  But,  fortunately,  waiting  for  water  to  boil  and  waiting  for  an  electric  stove  to 
heat  up  are  notorious  hassles.  By  putting  names  to  generic  hassles,  like  waiting ,  you 
provide  yourself  with  a  way  of  noticing  that  something  is  more  trouble  than  it  ought 
to  be. 

Associated  with  the  frames  for  many  generic  hassles  are  heuristics  for  repairing 
the  routines  they  occur  in.  (Brady  et  al  1984),  the  first  extended  exercise  in  applying 
the  theory  of  routines  to  a  real  world  domain,  describes  the  generic  repairs  for  some 
of  the  carpenter’s  generic  hassles: 

•  Insufficiently  controlled  degrees  of  freedom.  When  a  point  (like  a  screw  tip)  slides 
across  a  surface,  try  fixing  it  with  a  small  dent  (as  with  an  awl  or  gimlet).  When 
surfaces  slide,  increase  static  friction. 

•  Restricted  access.  The  nail  set,  the  offset  screwdriver,  and  the  angled  wrench  head 
represent  three  generic  plans  for  transmitting  force  into  tight  spaces. 

•  Changing  tools.  When  you  have  to  alternate  between  tools,  try  deinterleaving  the 
processes,  finding  a  combination  tool,  or  keeping  the  tools  as  handy  as  possible. 

•  Irreversible  overshooting.  If  you  take  off  too  much  with  a  saw,  you  can't  put  it 
back.  So  approach  the  desired  surface  with  successively  more  refined  methods, 
from  saw  to  plane  to  coarse  sandpaper  to  fine  sandpaper. 

Generic  hassles,  like  all  frames,  are  constantly  trying  to  instantiate  themselves  over 
whatever  is  happening.  When  one  of  them  succeeds,  it  brings  along  with  it  information 
that  might  be  useful,  in  the  form  of  warnings,  suggestions,  or  control  advice.  The 
insight  that  leads  to  identifying  generic  hassles  and  their  associated  generic  repairs 
also  leads  to  the  classification  of  other  kinds  of  generic  knowledge: 

A  fork  is  a  versatile  tool  that  is  at  once  an  instance  of  at  least  five  generic  models 
of  physical  structures:  flat  plate,  grating,  row  of  tines,  solid  block,  and  sharp  blade. 
The  most  important  generic  models  are  the  cognitive  cliches  (Chapman  forthcoming), 

thorough  domain  model  than  I  am  rowuming. 
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very  general  domain  independent  ideas  like  total  order,  propagation,  resource ,  and  en- 
abUment.  (The  theory  of  cognitive  cliches,  of  which  no  more  here,  directly  motivated 
roost  of  the  ideas  in  this  section.) 

Each  generic  model  brings  with  it  some  generic  plans.  To  scoop  something  up,  try 
moving  a  flat  plate  into  it  along  its  supporting  surface.  To  wash  out  a  grating,  try 
running  water  through  the  back  of  it.  To  retrieve  something  that’s  at  the  other  end 
of  a  cord  (a  generic  model  of  which  strings,  ropes,  wires,  and  occasionally  rolled-up 
table  cloths  are  instances),  try  reeling  in  the  cord. 

Routine  evolution,  as  I  mentioned,  is  the  internalization  of  process.  There  is  no 
general  way  to  characterize  a  process  one  is  engaged  in,  but  there  are  generic  processes 
covering  many  of  the  important  cases7.  I  have  mentioned  some  of  the  important 
generic  processes  already;  some  others  are:  consuming  a  resource,  approaching  a  goal 
(according  to  some  discrete  or  continuous  metric),  reaching  a  peak  (in  some  quantity), 
and  going  in  circles.  A  generic  process,  like  a  generic  hassle,  often  brings  along  advice 
about  how  to  modify  a  routine  it  describes.  If  you’re  waiting,  find  something  useful 
to  do.  If  you’re  consuming  a  resource,  be  careful  not  to  run  out. 

Lattices 

My  other  hypotheses  about  the  frame  network  that  supports  routinization  concern 
generalization  relationships  and  episodic  memory.  Formal  systems  with  relation¬ 
ships  of  generalization  and  specialization  tend  to  form  mathematical  lattices  (Birkhoff 
1967),  an  observation  made  as  early  as  1970  (Plotkin,  Reynolds).  That  they  do  so  is 
one  of  the  most  consequential  ideas  in  AI,  though  it  is  often  not  clearly  articulated. 
It  can  be  seen  underlying  work  on  analogy  (Winston  1980,  Gentncr  1980,  1983),  in¬ 
ductive  learning  (Winston  1975,  Mitchell  1983),  data  clustering  (Michalski  1980),  and 
classification  (Lipkis  1982,  Schmolze  and  Brachman  1982b). 

The  mental  latticework  describes  a  continuum  from  abstract  knowledge  to  con¬ 
crete,  weak  methods  to  strong,  general  concepts  to  specific,  large  extensions  to  small. 
1  suggest  that  people  put  a  lot  of  effort  into  maintaining  this  lattice.  Much  attention 
has  been  paid  to  the  most  abstract  parts  of  this  lattice,  whose  members  are  widely  ap¬ 
plicable  but  can  be  hard  to  apply  and  miss  fine  detail.  At  the  terminals  of  the  lattice 
is  episodic  memory,  literal  transcriptions  of  passing  reality.  By  storing  frames  along 
the  entire  continuum  of  intermediate  degrees  of  abstraction,  you  can  always  select  a 
frame  whose  degree  of  abstraction  is  suited  to  your  task.  The  generic  hassles,  models, 
plans,  and  processes  are  among  the  more  abstract  frames;  in  everyday  practice  one 
tends  to  use  special  cases  of  them. 

Part  of  the  job  of  episodic  memory  is  to  define  natural  classes  of  episodes  for  use 
in  future  analogical  reasoning.  There  are  (at  least)  two  ways  of  doing  this.  One  way 
is  to  look  for  pairs  of  episodes  that  are  broadly  similar  in  some  crude  terms  like  the 
proportion  of  shared  features.  The  class  of  episodes  that  shares  those  features  might 

7  Dyer's  (1984)  idea  of  a  TAU  is  very  similar.  In  Dyer’s  program,  though,  TAU’s  have  an  unfortunate 
modularity.  The  program's  knowledge  of  a  generally  useful  idea  (like  that  of  an  implicit  contract) 
is  embedded  in  a  description  of  a  particular  way  that  a  plan  using  that  idea  can  go  wrong. 
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be  a  useful  frame  to  define.  Although  1  imagine  that  many  of  the  scenes  in  the  simple 
script  representation  of  a  process  are  first  defined  this  way,  the  method  as  stated  is 
sorely  underconstrained,  even  under  the  assumption  of  massive  parallelism.  (Many 
clues  are  available  in  work  at  Yale,  see  Schank  1982.) 

A  more  satisfactory  way  of  defining  natural  classes  in  episodic  memory  is  called 
argument  indexing.  In  a  certain  situation  you  decide  to  act  in  a  certain  way.  If  you 
have  been  recording  your  reasoning  step  by  step  (in  the  manner  of  Doyle  1980),  then 
the  premises  of  that  reasoning  define  the  natural  class  of  episodes  in  which  you  would 
make  that  decision.  The  decision  to  conserve  a  resource,  for  example,  might  be  based 
solely  on  there  being  a  resource,  its  being  consumed,  and  there  being  a  danger  of 
running  out.  If  it  had  never  before  occurred  to  you  to  conserve  a  resource,  then  the 
generic  suggestion  of  conserving  a  scarce  resource  will  be  associated  with  the  resource 
cognitive  cliche. 

Most  decisions,  though,  are  based  on  finer  details  of  an  episode.  Suppose  you 
decide  to  rescue  a  burning  omelette  by  making  it  into  scrambled  eggs.  This  same  line 
of  reasoning  might  also  lead  you  to  stir  a  sauce  to  keep  the  sediment  from  burning, 
and  so  it’s  worthwhile  defining  a  frame  for  the  general  idea  of  the  bottom  surface  of 
a  heated  container  of  liquid  threatening  to  burn.  This  new  frame  would  contain  the 
generic  line  of  reasoning  about  stirring.  There  are  simpler  versions  of  the  argument 
indexing  algorithm  due  to  Rosenbloom  (see  Laird,  Roscnbloom,  and  Newell  1984)  and 
Minton  (1984).  Batali  (personal  communication)  is  building  a  theorem  prover  based 
on  a  very  similar  idea. 

!• 

Dialectical  argumentation 

The  original  motivation  for  this  work  was  (Doyle  1980).  Doyle  points  out  that,  by 
assimilating  thought  to  action,  a  system  can  support  domain  reasoning  and  meta¬ 
level  reasoning  in  a  uniform  manner.  To  achieve  this  end,  Doyle  proposed  the  method 
of  dialectical  argumentation s.  Whenever  there  is  a  decision  to  be  made,  any  active 
component  of  the  mechanism  is  free  to  adduce  arguments  as  to  how  it  should  be  made 
and  why.  When  there  are  disagreements,  deciding  among  the  arguments  is  itself  a 
decision  to  be  made  by  argumentation,  recursively:  each  mechanism  may  adduce 
counterarguments  as  to  why  its  arguments  are  better  than  the  others. 

Suppose  that  I  suggest: 

P9326:  (PROPOSE  (HOLD-UP  BAYBANKS)) 

Contradictory  arguments  are  put  forward: 

P9621:  (PROPOSE  (SUPPORT  (HOLD-UP  BAYBANKS)  MONEY- IN-IT)) 

P9442:  (PROPOSE  (OBJECT  (HOLD-UP  BAYBANKS)  IT-WOULD-BE-WRONG)) 

There  are  no  objections  to  accepting  the  first  argument,  and  so  it  is  accepted: 
P9818:  (TAKE  (SUPPORT  (HOLD-UP  BAYBANKS)  MONEY- IN- IT)) 

*  The  phrase  is,  of  course,  redundant.  Perhaps  dialectical  reasoning  would  be  better. 


8 


Routines 


But  some  part  of  the  rule  system  considers  the  second  argument  inferior,  and 
proposes  that  it  be  considered  so. 

P9600:  (PROPOSE  (OBJECT  (OBJECT  (HOLD-UP  BAYBANKS)  IT-WOULD-BE -WRONG) 

PREFER-PRACTICAL-TO-MORAL-ARGUMENTS) ) 

There  are  no  objections  to  that  line  of  argument,  so  it  is  accepted: 

P9710:  (TAKE  (OBJECT  (OBJECT  (HOLD-UP  BAYBANKS)  IT-WOULD-BE-WRONG) 

PREFER-PRACTICAL-TO-HORAL-ARGUMENTS) ) 

Consequently,  the  moral  argument  against  robbing  the  bank  is  rejected: 

P9466:  (BLOCKED  (OBJECT  (HOLD-UP  BAYBANKS)  IT-WOULD-BE-WRONG)) 

There  being  no  other  outstanding  objections,  the  motion  stands: 

P9336:  (TAKE  (HOLD-UP  BAYBANKS)) 

Dialectical  argumentation,  then,  involves  the  adducing  of  arguments  and  counter¬ 
arguments  concerning  a  proposal  for  action.  Because  each  argument  and  counterar¬ 
gument  is  itself  an  action,  the  argumentation  process  itself  is  fair  game  for  argumenta¬ 
tion.  The  system  approaches  the  problem  of  deciding  between  conflicting  arguments 
in  just  the  same  way  that  it  approaches  any  problem  in  the  world.  The  method  has 
some  important  properties: 

•  All  reasons  for  action  are  defeasible,  meaning  that  they  might  be  overridden  if 
there  are  good  reasons  to  do  so. 

•  The  decision  process  is  additive,  meaning  that  all  parts  of  the  system  can  con¬ 
tribute  to  the  reasoning  in  a  uniform  way,  by  contributing  arguments. 

•  Individual  decision  processes  are  automatically  compiled  into  TMS  networks,  so 
that  an  arbitrarily  complex  argument  structure  can  be  used  many  times  a  second. 
In  particular,  an  argument’s  conclusions  will  stay  in  as  long  as  its  premises  stay 
in. 

•  Because  a  compiled  argument  will  be  recapitulated  whenever  its  premises  are 
satisfied,  it  will  be  automatically  carried  over  to  analogous  future  situations. 

•  The  system’s  reasoning  can  be  modified  without  ever  introducing  a  side-effect.  By 
simply  providing  an  argument  explaining  why  an  action  is  a  mistake,  a  person  or 
program  can  arrange  for  it  to  be  overridden  in  the  future9. 


9  This  lark  of  Hide-effects  is  an  attempt  to  answer  one  of  the  lessons  of  Eurisko  (Leant  1983),  that 
even  well- motivated  modifications  to  the  running  source  code  can  wreak  substantial  havoc. 
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There  .ire  very  many  generic  arguments ,  among  which  are  such  cliches  as  do  it  now 
because  you  might  not  get  another  chance.  A  generic  line  of  reasoning  stored  in  a  frame 
is  implemented  as  a  pattern  of  argument  and  counterargument  that  has  prevailed  in 
some  context. 

In  the  domains  of  everyday  life,  it  is  common  for  every  candidate  action  in  a 
situation  to  have  drawbacks.  Consequently,  one  must  often  carry  out  an  action  in  the 
face  of  perfectly  valid  counterarguments.  A  hassle  is  a  pattern  of  such  conflict  that 
arises  repeatedly;  a  wish  is  very  often  formulated  as  a  desire  to  find  a  way  to  satisfy 
all  the  considerations  that  arise  in  such  a  conflict.  It  is  useful  to  classify  patterns  of 
conflict  and  their  means  of  resolution10. 


Running  arguments 

Over  the  last  few  months  I  have  developed  and  implemented  the  technique  of  running 

arguments.  Running  arguments,  an  idea  derived  loosely  from  (Doyle  1980),  are  a 

way  of  programming  an  agent  in  a  time-extended  world  that  has  several  important 

advantages  over  other  methods: 

•  Although  the  source  language  is  a  version  of  the  rule-based  language  Amord  (de 
Kleer  et  al  1978),  the  code  is  rapidly  compiled  into  TMS  networks  that  run  by 
propagating  simple  tokens  and  without  consing  (see  Doyle  1978). 

•  Both  the  rule  language  and  the  TMS  networks  are  well-suited  for  implementation 
on  massively  parallel  computers. 

•  The  language  supports  a  powerful  style  of  programming,  dialectical  argumenta¬ 
tion,  which  makes  possible  uniform  treatment  of  domain  reasoning  and  meta-level 
reasoning. 

•  As  time  passes,  incremental  changes  in  the  world  require  only  incremental  recom¬ 
putation  of  the  structure  of  the  arguments.  The  system  notices  only  differences 
that  make  a  difference. 

•  Because  the  system  per  se  lives  entirely  in  the  present  tense,  it  does  not  pre¬ 
judge  the  user’s  decisions  about  representations  of  time.  In  particular,  neither  the 
simulation  nor  the  decision  mechanism  impose  a  discrete  temporality. 

The  next  few  sections  describe  the  present  system  and  some  of  its  consequences. 

10  Research  into  the  conflicts  that  arise  in  reasoning  about  etliical  issues  (Kohlbcrg  1981;  see  also 
Gilligan  1982)  offers  suggestions  about  how  people  deal  with  conflicting  arguments.  Kogan’s  (1982) 
theory  of  Imman  understandings  of  self  and  other  assimilates  Kohiberg’s  stages  of  ethical  develop¬ 
ment  to  Piaget’s  stages  of  cognitive  development.  Kcgan’s  theory  of  the  mechanisms  underlying 
the  stage  transitions  bears  a  striking  resemblance  to  the  present  theory.  For  Kogan,  an  individual 
passes  into  each  new  stage  by  (as  I  would  put  it)  internalising  her  old  decision  processes.  My  use 
of  the  word  internalize  descends  from  psychoanalytic  theory,  which  speaks  of  people  internalising 
problematic  relationships  in  ail  attempt  to  get  control  over  them. 
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System  organization 

The  system’s  modularity  is  sketched  in  Figure  1. 

The  outer  loop  of  the  running  argument  system  moves  between  the  world’s  physics 
and  the  agent’s  reasoning.  The  system  is  built  on  a  memoizing  rule  system  called 
Life.  There  are  two  Life  databases,  one  for  the  world  simulation  and  one  for  the 
agent.  Numerical  computations  are  limited  to  a  small  Lisp  program,  the  physics 
model.  Here  are  the  steps  in  the  outer  loop: 

•  The  physics  model  computes  a  new  list  primitive  qualitative  world-properties. 

•  This  set  of  propositions  is  compared  to  the  set  from  the  last  cycle;  any  changes 
are  recorded  by  bringing  TMS  nodes  tn  and  out  in  the  world  simulation  database 
as  appropriate. 

•  This  causes  the  world  simulation  to  propagate  TMS  values  and,  occasionally,  to 
run  rules  to  cover  previously  unseen  cases.  This  process  incrementally  computes 
a  new  set  of  propositions  representing  the  agent’s  primitive  percepts. 

•  These  primitive  percepts  are  transmitted  to  the  agent  by  bringing  TMS  nodes  in 
and  out  in  the  agent  database  as  appropriate. 

•  The  agent  database  now  propagates  TMS  values  and,  occasionally,  runs  rules. 
This  happens  an  awful  lot  at  first,  and  less  and  less  as  time  goes  on.  This  pro¬ 
cess  incrementally  computes  a  new  set  of  propositions  representing  the  agent’s 
primitive  physical  actions. 

•  These  primitive  actions  are  transmitted  to  the  world  simulation  by  bringing  TMS 
nodes  in  and  out  in  the  world  simulation  database  as  appropriate. 

•  This  causes  the  world  simulation  to  propagate  TMS  values  and,  occasionally,  to 
run  rules.  This  process  incrementally  computes  (1)  a  new  set  of  judgements  about 
the  success  or  failure  of  the  agent’s  primitive  actions  and  (2)  a  new  set  of  propo¬ 
sitions  representing  primitive  qualitative  world-changes. 

•  The  physics  model  runs  again,  and  so  on  ad  infinitum. 

After  considering  an  example,  I  will  describe  the  workings  of  the  system  and  the  issues 
involved  in  programming  with  it. 

Example 

The  next  three  figures  show  some  simple  examples  of  the  program  running  in  a  simple 
blocks  world.  In  each  display,  the  left  panel  shows  commands  to  the  program  and  the 
statistics  it  computes  about  itself  on  each  cycle. 

(Notation:  M(R/T)  for  the  world  simulation,  the  agent  simulation,  and  the  total, 
with  M  =  number  of  calls  to  the  pattern  matcher,  R  =  number  of  rules  fired,  and 
T  =  number  of  propositions  whose  TMS  state  has  changed  from  in  to  out  or  out  to 
tn.  The  number  in  braces  is  the  maximum  depth  of  propagation  through  the  TMS 
network.  Lines  of  statistics  ending  in  a  “<”  record  the  workings  of  a  process  that 
attempts  to  keep  down  the  fanout  in  the  pattern  lattice  by  indexing  new  patterns.) 
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Figure  i.  Organization  of  the  runuing  argument  system.  The  running  argument  system  is  built 
on -a  rule-based  language  called  Life.  The  system  itself  is  organized  around  a  loop  of  incremental 
updates  of  the  physics  model  (written  in  Lisp),  the  world  simulation  (written  in  Life  rules),  and  the 
agent’s  reasoning  (also  in  Life  rules).  The  agent’s  rule  sot  is  organised  in  three  parts,  a  general- 
purpose  control-si ruct lire  package  based  on  Auiord’s  techniques  for  explicit  control  of  reasoning,  a 
small  collection  of  rules  .supporting  dialectical  argumentation,  tuul  the  domain-specific  arguments  and 
nieta-argiimeiils. 
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In  Figure  2,  I  ask  the  system  to  put  B  on  C.  On  the  first  tick,  proposals  are  made 
to  move  the  hand  to  the  left  and  down,  on  grounds  of  getting  closer  to  the  block. 
There  are  no  objections.  On  the  next  tick,  there  is  no  more  reason  to  go  left  because 
the  hand  is  over  the  block,  so  only  the  argument  for  going  down  goes  through.  By 
the  third  tick,  there  are  no  new  considerations,  and  so  the  hand  keeps  moving  down 
without  any  rules  being  run,  until  finally  it  comes  in  contact  with  the  block. 

Figure  3  traces  the  progress  of  the  hand  as  it  carries  block  B  toward  block  C. 
Initially  it  moves  both  left  and  up,  since  both  directions  make  forward  progress.  But 
as  block  B  makes  contact  with  the  side  of  block  C,  another  rule  objects  to  moving 
left  on  the  grounds  that  it  will  move  C  unnecessarily.  So  the  hand  and  B  move  up 
for  several  cycles.  Finally  they  move  to  the  left  by  one  when  B  clears  C  and  there  is 
no  more  reason  to  go  up.  In  this  blocks  world,  blocks  are  sticky  enough  that  just-on 
counts  as  on. 

Figure  4  shows  the  results  of  two  experiments.  The  world  is  restored  to  its  original 
state  and  the  same  task  is  given.  This  time,  the  same  lines  of  reasoning  are  carried 
out  entirely  in  the  TMS,  with  the  exception  of  a  couple  of  stray  rule  firings  that 
result  from  the  discontinuous  world-change  when  the  original  state  was  restored.  In 
the  second  experiment,  B  and  C  are  interchanged  and  the  task  of  putting  C  on  B  is 
posed.  Even  though  this  task  is  isomorphic  to  the  last  one,  many  rules  must  be  fired, 
though  not  as  many  as  for  the  first  task..  Some  of  the  compiled  reasoning  has  been 
carried  over  to  the  new  task,  but  not  as  much  as  I  would  like.  The  problem  is  that 
the  rules  refer  to  blocks  by  their  names,  so  that  reasoning  carried  out  about  A  and  B 
can  only  be  carried  over  to  B  and  C  by  substituting  variables  for  the  names.  Finding 
more  generalizable  names,  or  substitutes  for  names,  is  a  current  project. 


The  Life  rule  system 

Life  is  a  simplified  variant  of  the  rule-based  system  Amord.  The  Life  system  has  three 

tightly  intertwined  components: 

•  There  is  a  lattice  for  storing  patterns.  Every  pattern  in  the  lattice  is  connected 
to  those  other  patterns  that  are  immediately  more  general  than  and  immediately 
more  specialized  than  them.  See  Figure  5. 

•  There  is  a  TMS  that  constantly  maintains  a  collection  of  justification  relationships 
among  the  patterns.  When  a  pattern  is  made  in  or  out ,  the  effects  of  this  change 
are  propagated  throughout  the  network  of  justifications.  See  Figure  6. 

•  There  is  a  rule  system  that  operates  over  the  lattice.  A  rule  is  stored  on  the  pattern 
corresponding  to  its  left  hand  side11.  When  a  more  specialized  pattern  is  made  in 
by  the  TMS,  the  rule  fires,  constructing  the  appropriate  TMS  justification  for  the 
appropriate  consequent. 


11 1  heard  about  this  scheme  from  David  Chapman. 


Figure  2.  Please  put  B  on  C.  On  each  turn  of  the  world  clock,  stutintirs  show  roughly  how  much 
work  the  program  w  doiug.  Rule*  only  need  to  he  fired  wlien  novel  mtuntious  are  enroitntercd,'  once 
the  program  luu*  jm-oii  a  VJiriety  of  siituntioii.s  all  the  work  Mturta  to  l»e  done  in  the  TM8. 
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Figure  3.  Moving  D  toward  C.  The  motion  w  not  especially  graceful.  moving  m  fart  nn  piwiblc  along 
ourli  dimension  separately.  Learning  to  integrate  arguments  alxiut  moving  vrrtirally  with  arguments 
about  moving  horizontally  is  a  future  task  for  the  program. 
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Fii'iiri'  4.  Two  experiment!.  Repeating  the  tank  of  putting  D  on  C  requires  only  a  trivial  amount  of 
wiirk  outside  the  TMS.  However,  the  isomorphic  tank  of  putting  C  on  D  in  a  world  where  they  started 
out  in  exrhniigril  plnres  requires  many  rub’s  to  he  liml,  not  as  niiuiy  b«it  still  too  tunny. 
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Figure  5.  The  pattern  lattice.  Patterns  are  organised  in  a  lattice,  with  the  most  inclusive  at  tin*  top 
and  the  most  specific  at  the  bottom.  The  primary  advantage  of  litis  scheme  is  that  all  of  the  instances 
of  a  jMtleru,  or  all  of  the  patterns  generalising  nil  instance,  can  be  found  ipiickly  by  a  simple  lattice 
traversal. 


/VoT^  riCrJ  ; 


(  PW  77  rnt:  )  (cUkJpy  0^/06  ) 


Figure  C.  The  TMS.  Every  proposition  that  1ms  ever  been  in  lms  a  list  of  justification*.  For  a 
justiKeatiou  to  provule  support  for  a  proposition,  all  the  propositions  on  its  in  list  must  be  in  anil 
all  the  propositions  on  its  out  list  must  be  out.  Whenever  a  proposition  changes  state,  a  potentially 
extensive  updating  process  begins  to  pro|>agntc  through  the  network  of  propositions  anil  justifications. 
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The  Life  language  has  two  primitive  forms,  if  and  unless ,  whose  semantics  is  not 
imperative  but  declarative. 

PI:  (if  (loves  ?x  ?y)  (likes  ?x  ?y)) 

“So  long  as  ?z  loves  ?y,  lx  likes  ?y.” 

P2:  (if  (person  ?y)  (unless  (loves  ?x  ?y)  (lonely  ?y))) 

“So  long  as  nobody  loves  ?y,  ?y  is  lonely." 

If 


P3 :  (person  george) 

is  asserted,  then  so  are: 

P4:  (unless  (loves  ?x  george)  (lonely  george)) 

PS:  (lonely  george) 

Now  PA  is  justified  in  terms  of  P2  and  PS,  and  P5  is  justified  in  terms  of  PA.  If, 
now, 

P6:  (loves  sam  george) 

is  asserted,  then 

P7:  (likes  sam  george) 

is  also  asserted.  Because  PI  matches  the  left  hand  side  of  PA,  the  justifications  of 
all  of  PA's  past  consequents  are  changed  to  include  P7  on  their  out-list.  Thus  P5, 
having  lost  its  TMS  support,  is  retracted.  See  Figure  7. 

Despite  its  simple  primitive  functionality,  Life  is  nonetheless  a  powerful  program¬ 
ming  system  because  its  rules  are  asserted  in  the  same  database  as  its  other  proposi¬ 
tions.  This  allows  rules  to  fire  on  other  rules.  Here,  for  example,  are  the  rules  that 
support  and  clauses  in  the  left  hand  sides  of  rules  and  progn  clauses  in  the  right  hand 
sides: 

(If  (if  (and  .  ?p)  ?q) 

(build -and  .  ?p)) 

(if  (unloas  (and  .  ?p)  ?q) 

(build-and  .  ?p)) 

(if  (build-and  ?p  ?q  .  ?r) 

(if  ?p  (if  (and  ?q  .  ?r)  (and  ?p  ?q  .  ?r)))) 

(if  (build-and  ?p)  (if  ?p  (and  ?p))) 

(if  (progn  ?p  .  ?q)  ?p) 

(if  (progn  ?p  .  ?q)  (progn  .  ?q)) 

This  makes  it  easy  to  write  rules  of  this  sort: 


10 
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Figure  7.  An  unless  rule  has  been  overridden. 
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(if  (block  ?block) 

(progn  (if  (and  (on  ?block  ? support) 

(supported  ?support  ?root)) 

(supported  ?block  ?root)) 

(if  (grasping  ?block) 

(supported  ?block  hand)) 

(unless  (supported  ?block  ?root) 

(unsupported  Tblock)) 

(if  (unsupported  ?block) 

(falling  ?block)))) 

Amord  techniques  for  the  explicit  control  of  reasoning  are  easily  supported  as  well: 

(if  (if-shovn  ?p  ?q) 

(progn  (try-to-show  ?p) 

(if  ?p  ?q))) 

(if  (try-to-show  (and  ?p  ?q  .  ?r)) 

(progn  (try-to-show  ?p) 

(if  ?p  (try-to-show  (and  ?q  .  Tr))))) 

(if  (try-to-show  (and  ?p))  (try-to-show  ?p)) 

(if  (try-to-show  (and  ?p  .  ?r))  (build-and  ?p  .  Tr)) 

See  (de  Kleer  et  al  1977)  for  further  explanation  of  this  sort  of  rule  language  pro¬ 
gramming. 

Here  are  the  Life  rules  underlying  the  bank  robbery  example: 

(def rules  thats-where-the-money-is 

(can-show  (practical-argument  money-in-it)) 

(can-show  (moral -argument  it-would-be-wrong)) 

(propose  (hold-up  baybanks)) 

(can-show  (profitable  (hold-up  ?anything))) 

(can-show  (wrong  (hold-up  ?anything))) 

(if  (propose  Taction) 

(progn  (if-shown  (profitable  Taction) 

(propose  (support  Taction  money-in-it))) 

(if-shown  (wrong  Taction) 

(propose  (object  Taction  it-would-be-wrong))))) 

(if  (and  (propose  (support  Taction  Tpractical-argument)) 

(propose  (object  Taction  Tmoral -argument))) 

(if-shown  (and  (practical-argument  Tpractical-argument) 

(moral -argument  Tmoral -argument)) 

(propose  (object  (object  Taction  Tmoral-argument) 

prefer-practical-to-moral-arguments))))) 

Dialectical  argumentation  is  supported  by  a  deceptively  simple  set  of  rules: 

(defrules  interpreter 
(if  (propose  Taction) 

(unless  (blocked  Taction) 

(take  Taction))) 

(if  (take  (object  Taction  Targument)) 

(blocked  Taction))) 


I  arrived  at  this  formulation  of  argumentation  in  the  course  of  trying  to  understand 
Doyle’s  (1980)  discussion  of  the  problem  of  writing  a  fully  reflexive  interpreter.  A 
decision  method  is  fully  reflexive  if  its  every  detail  is  capable  of  being  questioned  and 
potentially  overridden  in  the  light  of  new  discoveries  or  preferences. 

One  way  to  try  making  a  decision  method  reflexive  is  to  require  it,  before  it  makes 
the  slightest  move,  to  pose  itself  the  problem  of  deciding  whether  it  ought  to  make 
that  move.  But  this  leads  immediately  to  an  infinite  regress:  the  act  of  deciding 
whether  to  move  ought  to  be  decided  on  itself,  which  deciding  ought  to  be  decided 
upon  in  turn,  and  so  on.  It  is  not  at  all  obvious  a  priori  that  it  is  possible  to  build 
a  reflexive  decision  method  that  does  not  get  itself  into  infinite  regresses  (see  Batali 
1983). 

Doyle’s  hypothetical  SEAN  program  (1980)  employs  the  following  algorithm: 

Top  level  loop: 

Do  forever 

Figure  out  what  to  do 
Do  it 

To  figure  out  what  to  do: 

Enumerate  candidate  actions 
Adduce  arguments  for  or  against 
If  all  arguments  favor  one  action 
then  Take  that  action 

else  Recursively  figure  out  how  to  settle  the  argument 
Take  the  winning  action 

This  scheme  suffers  from  three  important  difficulties: 

•  The  decision  mechanism,  by  only  deciding  upon  and  performing  one  action  at  a 
time,  tends  to  impose  a  discrete  model  of  time  and  an  atomic  model  of  action. 

e  The  decision  mechanism  is  too  highly  centralized.  Often  it  is  useful  for  different 
non-interacting  decision  processes  to  go  on  in  parallel. 

•  There  is  no  clear  way  to  give  up  on  a  decision  process  when  the  arguments  don’t 
render  a  clear  decision.  Both  Minsky’s  Law  of  Non-compromise  and  practical 
experience  indicate  that  this  is  often  the  most  nature  move. 

The  difference  between  Doyle’s  mechanism  and  mine  is  in  the  mechanism  of  arbitra¬ 
tion.  If  there  are  two  proposals  for  action,  SEAN  automatically  invokes  a  process  to 
decide  between  them.  My  mechanism,  by  contrast,  only  attempts  to  arbitrate  be¬ 
tween  proposals  for  action  if  someone  notices  a  conflict  and  makes  an  objection.  This 
way,  many  processes  of  reasoning  and  acting  can  proceed  asynchronously  in  parallel. 
Actions  are  carried  out  so  long  as  they  have  TMS  support. 

Scenario 

It  will  help  in  making  my  descriptions  concrete  to  consider  a  scenario  that  I  would 
like  the  finished  system  to  be  able  to  limp  through. 
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Imagine  a  simple  difference  reducing  planner  facing  the  task  of  producing  a  bowl 
of  cereal  with  milk  and  bananas,  with  a  spoon  to  one  side.  The  program  comes  with 
a  library  of  operators  for  reducing  differences,  with  add  lists  and  delete  lists  in  the 
manner  of  Strips  (Fikes  and  Nilsson  1971).  The  program  does  not  know  some  basic 
things,  as  will  become  evident. 

Making  a  bowl  of  cereal  is  a  substantial  feat  if  you  haven’t  even  got  a  routine 
for  bringing  the  box  of  cereal  over  to  the  table.  The  first  routine  that  develops  is 
the  routine  for  deciding  to  bring  the  box  of  cereal  to  the  table.  The  process  to  be 
captured  is  entirely  internal,  involving  an  enumeration  of  differences  to  be  reduced  and 
a  process  of  arguing  out  which  one  to  reduce  first.  The  program  makes  an  instance  of 
the  generic  argument  process  for  this  particular  argument.  The  next  time  the  issue 
comes  up,  the  argument  will  proceed  differently,  because  this  new  frame  will  provide 
a  really  compelling  argument  about  why  getting  the  cereal  is  the  right  thing  to  do 
next. 

Actually  getting  the  cereal  is  a  different  matter.  You  can’t  put  the  cereal  down 
because  you  haven’t  got  it  in  your  hand,  and  you  can’t  pick  it  up  because  it’s  in 
the  cupboard  across  the  room  and  because  the  cupboard  is  closed,  and  so  on.  The 
system  improvises,  repeatedly  assessing  the  situation,  selecting  a  difference  to  reduce, 
and  reducing  it.  There  will  arise  a  stereotyped  process  of  walking,  thinking,  opening, 
thinking,  picking  up,  thinking,  walking,  thinking,  and  putting  down.  The  system 
will  build  a  representation  of  this  process,  in  about  two  stages.  First  the  program 
will  write  down  the  component  processes  of  thinking  and  doing.  Then  it  will  notice 
that  these  processes  often  occur  end-to-end  and  write  that  down  too.  Because  the 
reasoning  that  goes  into  these  processes  is  independent  of  it  being  a  cereal  box  rather 
than  a  spoon,  say,  or  a  cupboard  instead  of  a  refrigerator,  the  process  descriptions 
will  be  generalized  appropriately. 

So  when  it  comes  time  to  reduce  the  difference  of  there  being  no  spoon,  or  no  milk, 
a  series  of  really  compelling  suggestions  will  appear  about  how  to  go  about  getting 
them. 

Some  generic  process  concerned  with  iteration  will  become  interested  in  this  se¬ 
ries.  One  of  the  things  you  know  about  iterated  processes  is  that  they  often  involve 
overhead  that  can  be  eliminated  by  stringing  together  the  content  of  each  iteration 
(see  Waters  1978,  1979).  So,  one  day,  after  the  program  has  picked  up  the  cereal  box, 
it  will  occur  to  it  to  get  the  spoon  too  while  it’s  at  it.  Once  it  has  the  spoon,  it  will 
occur  to  it  to  get  the  bowl  too.  And  so  on.  Acting  on  these  suggestions  will  itself 
become  the  subject  of  new  process  descriptions. 

The  route  that  our  imaginary  robot  follows  when  fetching  box,  spoon,  bowl,  and 
milk  will  interest  generic  processes  concerned  with  the  following  of  paths.  Such  process 
descriptions  can  often  offer  advice  on  the  order  in  which  different  points  on  a  path 
ought  to  be  visited.  This  advice  will  change  the  outcome  of  each  successive  argument 
about  which  object  to  fetch  next. 

If  the  robot  has  only  two  hands,  it  will  become  annoyed  when  its  attempt  to  pick 
up  the  third  object  fails.  With  some  care,  it  ought  to  be  possible  to  arrange  for  it  to 
see  the  problem  as  one  of  insufficient  carrying  capacity.  There  are  two  important  ideas 
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that  come  with  the  notion  of  capacity.  They  are  making  several  trips  and  increasing 
the  capacity.  The  program  will  wonder  if  there  is  a  way  to  increase  one’s  carrying 
capacity  and  notice  that  this  is  exactly  what  a  tray  is  for.  The  next  time  around,  the 
ideas  of  making  several  trips  and  using  a  tray  will  both  occur  to  it. 

There  is  no  good  way  to  tell  which  of  these  two  options  is  better  a  priori,  so  the 
decision  will  be  underconstrained  and  made  arbitrarily.  Making  an  arbitrary  choice  is 
a  generic  process.  One  of  the  generic  arguments  associated  .with  arbitrary  choices  is 
try  the  other  one  this  time  to  see  what  it ’s  like.  The  typical  breakfast  maker  would  take 
this  advice  and  decide  between  the  two  options  on  the  basis  of  which  one  involves  less 
hassle.  I  have  not  been  able  to  learn  much  about  how  people  make  such  comparisons. 

An  enormous  variety  of  issues  bear  on  the  remaining  steps  in  preparing  a  bowl 
of  cereal,  but  representing  them  is  likely  to  be  too  hard  for  a  long  time.  Especially 
complex  is  the  reasoning  that  goes  into  slicing  a  banana  onto  a  bowl  of  cereal.  Peeling 
the  banana  two- thirds  of  the  way  down  is  better  than  peeling  it  completely  right  away 
because  it  gets  your  hands  less  messy.  When  cutting  the  slices,  passing  the  knife 
through  from  the  bottom  of  the  banana  is  better  than  passing  it  through  from  the 
top.  The  reason  is  that  the  slices  of  banana  stick  to  the  knife;  striking  the  banana 
from  below  knocks  the  stuck  slice  straight  down  into  the  bowl  rather  than  making  it 
roll  off  the  top  of  the  banana.  The  right  thickness  for  the  slices  is  the  one  that  lets  you 
cover  the  top  of  the  cereal  evenly.  The  slices  are  usually  too  big  to  easily  leave  enough 
room  on  the  spoon  for  cereal.  One  way  to  deal  with  this  is  to  cut  the  banana  once 
lengthwise  after  peeling  it  and  before  slicing  it.  Some  people  save  washing  a  utensil 
by  slicing  their  bananas  with  their  spoons.  Most  everyone  who  does  this  thinks  they 
invented  it. 

Proposal 

As  I  have  been  describing,  I  have  by  this  time  established  most  of  the  theory  and 
the  beginnings  of  an  implementation.  My  thesis  will  present  a  (presumably  small) 
number  of  examples  of  the  program  improvising  new  routines  and  then  modifying 
them  in  the  light  of  experience.  Of  the  many  dimensions  of  the  space  of  possible 
examples,  I  am  primarily  concerned  with  two: 

•  Although  the  current  toy  examples  are  taken  from  blocks  world,  this  is  only  a 
reflex  on  my  part.  The  theory  of  routines  is  intended  to  explain  what  goes  on 
in  domains  more  representative  of  everyday  life.  I  am  unlikely  to  have  the  time 
or  technical  wherewithal  to  connect  the  program  to  real  cameras  and  robots,  so 
any  “real”  domain  will  be  entirely  artificial.  But  even  an  artificial  scenario  like 
that  of  the  previous  section  will  serve  to  demonstrate  both  the  unsuitability  of 
classical  domain-independent  planning  and  the  alternative  provided  by  the  theory 
of  routines. 

•  My  real  interest  in  routinization  comes  from  its  connection  to  the  theory  of  per¬ 
sonality.  People  construct  their  minds,  very  roughly  speaking,  by  internalizing 
their  interactions  with  their  environments.  In  real  life  this  requires  many  cycles 
of  internalizing  an  old  pattern  of  activity  and  watching  as  a  new  one  emerges. 
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It  is  possible  that  the  program  will  hold  together  for  enough  cycles  to  construct 
for  itself  the  basics  of  classical  linear  planning.  Naturally  this  will  require  some 
cheating. 

The  implementation  needs  several  more  parts.  As  it  stands,  it  is  terribly  myopic.  It 
can  muddle  through  most  simple  blocks  world  situations,  but  its  learning  is  confined  to 
TMS-compilation  of  its  existing  rules.  It  has  yet  to  internalize  any  processes,  though. 
The  next  few  modules  will  manipulate  process  representations,  called  scripts.  A  script 
is  an  interval-based  representation  of  a  pattern  of  activity  (Allen  and  Koomen  1983). 

•  The  interface  between  the  running  argument  system  and  the  script  database  is 
the  chart  recorder,  largely  written.  Certain  propositions  (selected  somehow)  are 
monitored.  Imagine  each  proposition  dragging  a  pen  behind  it  on  a  moving  scroll 
of  paper,  as  in  a  real  chart  recorder.  When  the  proposition  goes  in,  the  pen  goes 
down,  and  when  it  goes  out  the  pen  goes  up.  The  resulting  pattern  of  line  segments 
is  matched  against  the  scripts  in  the  database. 

•  Another  module  maintains  the  script  database.  The  hardest  part  of  this  module 
is  the  script-subsumption  algorithm,  which,  given  two  scripts,  determines  if  one 
extensionally  includes  the  other.  The  complexity  of  this  algorithm  will  limit  the 
extensions  I  can  make  to  the  script  representation;  I  need  in  any  event  some 
notation  for  iterated  processes. 

•  The  database  will  start  out  with  a  collection  of  generic  scripts.  Many  of  these  will 
have  arguments  to  offer  when  they  are  recognized;  others  will  cause  the  program 
to  hypothesize  that  some  generic  model  applies  to  the  relevant  elements  of  the 
domain.  Learning  to  write  this  code  will  be  the  primary  interest  of  the  exercise. 

•  The  worst  part  of  the  entire  program  is  the  module  that  attempts,  by  whatever 
means,  to  synthesize  new  commonly-recurring  scripts  from  the  program’s  con¬ 
tinuing  history  of  interaction  with  its  world.  I  don’t  expect  to  be  proud  of  the 
algorithm. 

This  shouldn’t  take  much  more  than  a  year. 
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